'''

    DataK

CONCEPT
    DataK is a tree structured relational database.
    Data are represented by items belonging to a connections.
    
    ## ITEM ##
    Each item has an id, a name, a value, and some attributes:
        # id #
        It is unic for the item's connection.
        It represents the tree structur of the items:
            "ENUM/STATUS/WIP" is a child of "ENUM/STATUS"
        
        # name #
        It is the last part of the id (aka basename)
        
        # value #
        The item's value defaults to its name and can be overriden
        by the attribute 'value'
        
        # attributes #
        Attributes are names associated to values (which defaults to None).
        * Attribute Names *
        The names must be valid python names without leading underscores:
            "date"      OK
            "_date"     Bad!
            "1_thing"   Bad!
            "thing_1"   Ok
        A trailling underscore in the name makes the attribute inherited
        from the item's parents. That is, its value will be None only if
        nothing defines this name in the item's path from the connection.
            "date"      local attribute
            "date_"     local attribute or herited if not defined.
        
        * Attribute Values *
        The value can be anything which builds itself by executing its 
        repr() string.
        When setting with a strings starting by '=', the value
        become an expression. This expression is stored "as is" and the value
        will be the result of the evalution of this expression by the item.

        
    Items are accessibles thru the connection, parenting relations
    and by search queries:
        # from connection
        If you know the item's id, the connection can give you the item.
        # from parenting relations
        If have the parent item, it can give you any of its children.
        # from search queries
        Every item can search its children for some ids matching a wilcard
        and/or attributes matching an expression.
        The result of the search is alway a Group object.
    
    ## GROUP ##
    A group is a collection of items and is actually an item itself.
    * Setting an attribute on the group does it on all items.
    * Getting an attibute on a group returns None until all items have
    the same value for this attribute (which can still be None)
    * A group has an empty id.
    * A group acts like if every contained items where its chidlren.
    * Thus, searching from a group aggregate the results of the same search
    on every contained items.
    
    ## ORDER ##
    When accessing item's children, order matters.
    To control it you can give to each items an expression used to sort
    its children when acting on them (access, search, etc...).
    
    ## SPLIT ##
    Items can be splitted into several groups using an expression.
    Items giving the same result will be distributed in the same group.
    
    ## LINK ##
    The relations (other than the parenting one) are represented by
    links.
    A link is as item who's attributes are actually the attributes of
    another item, execpt for its id (and thus its name).
    A link has an special attribute 'targetId' which is the id of the
    item which actually owns the attributes.
    
    ## Connection ##
    It is responsible of Item instanciation and on disk localisation.
    If it has a ON_START attibute, it will be executed right after creation.
    If it has a ON_QUIT attibute, it will be executed right after deletion.

UNDERNEATH
    You need to deal with 4 Classes:
        Connection, Item, Link, Group
    
    ** The connection is a singleton on the pair id/user.
    It's id is its path on disk.
    It resolves Item paths like this:
        os.path.join( connection.id, item.id )
    It is reponsible to create Item or Link instances.
    
    ** An Item is a singleton on the pair connection/id.
    You never instanciate an Item, but request it from
    a Connection.
    The Item class represents a folder on the file system. 
    Attributes of an item represent files in the folder of the
    item. The attribute name is the file name, and the attribute 
    value is the file content (expressed in python).
    When overriding the Item's value, you actually write to 
    an hidden attibute named ".value".
    You can iterate on the Item like on a list and request children
    by name like on a dict.
    You can search descendants of an Item with wilcarded ids and 
    match expression. Seach return a Group.
    Sorting an item affects the way it behaves with his children.
    This means changing the sort has no cost, but complex sort
    make search slower.
    Spliting an item is a short for searching all children and split
    the result. 
    
    ** A Link is an Item pointing to another Item. 
    You never instanciate an Link, but request it from
    a Connection.
    The Link class represents a link on the file system.
    Its value, attributes, and children are taken from 
    the target Item.
    The Link uses the special attribute "targetId" to store 
    the id of the target item.
    
    ** A Group is Item with an empty id. It is NOT a singleton.
    You can create groups at will.
    Its children are the content of the group.
    Sorting a Group reoders its content and affects the way it
    behaves with his children. This means the cost is immediate AND
    occurs on further searches. It is a best practice to sort the Item
    before searching rather than searching and then sort the group.
    Splitting a Group returns a dict with value as key Group as value.
    
SYNTAXE
    ## ITEM VALUE ##
    # get the value of item
    item       #=> evaluate to 50
    print item #=> evaluate to str(50)
    item == 50 #=> True
    item + 10  #=> 60
    10 + item  # will not work, you must use the long style:
    10 + item.value
    
    # set the value of item
    item = 20
    item += 10 # deprecated! operators ending with '=' are reserved
    # to tune Item behavior.

    ## ATTRIBUTES ##
    # get the attribute 'attr' of item:
    print item.attr

    # test if an attribute 'attr' exists in item:
    item.attr is not None
    
    # get the herited attribute 'attr' of item:
    print item.attr_
    
    # create the attribute 'attr' in item:
    item.attr = 50

    # change the value of the attribute 'attr' of item:
    item.attr = 50
    item.attr += 50
    item.attr *= 2

    # set the expression of the attribute 'attr' of item:
    item.attr = '=(50+50)*2'                # simple python code
    item.attr = '= sum( @TASK/*.length )'   # using referenced ids

    # delete the attribute 'attr' of item:
    item.attr = None

    ## SORTING ##
    # tell item to process children in the order of an attribute
    item << 'start_timestamp'
    
    # use an expression to compare processed children:
    item << '@STATUS/COMPO.asInt'
    
    ## SPLITTING ##
    # Splitting an Group give a dict with distinct values of the
    # given expression for each contained Item:
    byLength = shots / '@RANGES.end - @RANGES.start + 1'
    for length, shots in byLength.iteritems():
        print 'Shot with', length, 'image(s)':
        print [ str(shot) for shot in shots ]
        
    # Splitting an Item is a shortcut for:
    grouped = item['*'] / 'attrName'
    
    ## CHILDREN ACCESS ##
    # by list comprhension:
    children = [ i for i in item ]
    
    # by index:
    child = item[5]
    someChildren = item[1:]
    someChildren = item[1:12:3]
    allChildren = item[:] # returns a list
    
    # by child name:
    toto = item['TOTO']
    
    # by relative id:
    totoStatus = item['TOTO/STATUS']
    parent = item['..']
    
    # by absolute id:
    seq = anyItem['/FILM/S0200']
    
    # there is a shortcut to test one children value:
    50 in item <==> 50 in item / 'value' <==> 50 in item['*'] / 'value'

    ## ALL DESCENDANTS ACCESS ##
    # Use a wilcarded id to get a Group:
    allTaks = item['TASKS/*']
    
    # Add an expression to filter the results:
    myTasks = item['TASKS/*':'@.len > 3']
    
    # if an expression is given without id, the id defaults to '*':
    item[:'@.len > 3'] <==> item['*':'@.len > 3']

DATAK SPECIFIC SYNTAXE
    ## WILCARDED IDS ##
    # you can use * and ? in the wilcarded ids:
    allShots = sequence['P*']
    allShots = sequence['P???']
    firstTenShots = sequence['P1??']
    
    # wildcards ids can be relativ or absolute:
    item['TASK/*.user']
    item['../TASKS.user']
    item['/*/TASKS.user']

    # An empty id returns the item itself.
    # (It is not a wilcarded, no Group is built here)
    item == item['']

    ## EXPRESSION ##
    # Expressions are used to filter searches, order items and groups, and
    # split items and groups.
    # Expressions are python code optionnally using reference ids.
    # a reference id is made of a '@' folowed (or not) by wilcarded (or not) id.
    #   '@'             -> reference to the current item.
    #   '@TASK/COMPO'   -> reference to the compo task of the current item.
    #   '@TASK/*'       -> reference to all tasks of the current item.
    # An expression consiting of a single word is a shortcut for the 
    # attribute name by this word in the current item.
    #   'date' <==> '@.date'
    # Expressions used to sort and group can't use wildcards in reference ids
    # as it would cost expensive work to get uninteresting results.
    
    # Simple evauation of an expression:
    length = item( "@RANGES/ANIM.end - @RANGES/ANIM.start + 1 " )
    
    # example expressions with sort:
    item << 'date' <==> item << '@.date'
        -> sorts on the value [ i.date for i in item ]
    
    item << '@TASK/COMPO.start_date'
        -> sorts on the value [ i['TASK/COMPO'].start_date for i in item ]
    
    # example expression to filter searches:
    item[:'@.attr > 10'] <==> Group( [ i for i in item if i.attr > 10 ] )
    item['sub/*':'@.attr > 10 and @child.attr > 2']
        <==> Group( 
            [ i for i in item['sub/*'] if i.attr > 10 and i['child'].attr > 2 ]
            )
            
    # complexe one:
    item['SUB*':'50 in @SUBSUB["Z*"]']
        <==> selects the children of item with a name starting by 'SUB'
        and having a child named 'SUBSUB' if this child has at least one
        child with a name starting by 'Z' and having a value of 50.
        Pffiuu ! ^^
    
<< Select >>
    # what does this return ? a group ? a table ?

    # local attribute(s)
    item['attr']              <==> item.attr
    item['name','id','color'] <==> [item.name, item.id, item.color]
    
    # relative/absolute attribute(s)
    item['attr':'*']          <==> [ i.attr for i in item.Exec('*') ]
    item['id','name':'c*']    <==> [ (i.id,i.name) for i in item.Exec('c*') ]
    item['attr':'/root/*']    <==> [ i.attr for i in item.Exec('/root/*') ]

    # local attribute with a where clause 
    item['name'::'@sub.attr > 10']
        <==> [ i.name if i.Get('sub').attr > 10 ]

    # relative and/or herited attributes with a complex where clause
    item[
        'name','color_'
        :'c*'
        :'@.active and @sub["user","status"] == ["bob","WIP"]'
    ]
        <==> [
                (i.name, i.color)
                for i in item.Exec('c*')
                if i.active 
                and i.Get('sub')["user","status"] == 'bob'
             ]

'''

#-#######################
#--- syntax proposal
#-#######################

# connect
cnt = Connection( serverPath, login )

# access root items:
roots = [ i for i in cnt ]          # build a list of Item
roots = cnt[:]                      # get a Group
roots = cnt['LIB*']                 # get a Group
roots = cnt[1:-1:2]                 # get a Group
LIB = cnt['LIB']                    # get one root Item

# item attributes
v = LIB.attrName                    # get local attribute
v = LIB.attrName_                   # get herited attribute
LIB.attrName = v                    # assign to local attribute
LIB.attrName_ = v                   # raise AttributeError

# item special attributes
LIB                                 # get the value of the item
LIB.value = v                       # set the value of the item
LIB.id                              # get the id (path under connection)
LIB.path                            # get the abs path

# select from an item
CHARS = LIB['CHARS']                # get one Item
families = LIB['*']                 # get a Group
allChars = LIB['CHARS/*']           # get a Group
myCharFx = LIB['CHARS/*':'@TASKS/FX.user == dee'] # get a Group

# sorting a item
LIB << 'name' << 'date'             # configure the Item
sorted = [ i for i in LIB ]         # build a sorted list
sorted = LIB['*']                   # get a sorted Group

# divide by common value (grouping)
group = LIB['CHARS/*':'@TASKS/*/STATUS == "WIP"']
grouped = group / '@TASKS/*.user'   # get a dict{value:Group}
for value, grp in grouped:
    print 'Tasks for user', value,':'
    for item in grp:
        print ' ', item

# create an item
item = cnt.Create( id='/path/to/item' )  # create an new item
item2 = cnt.Create( id='/path/to/item' ) # creation skiped...

# create a link
cnt.Create( id='/path/to/item', targetId='/path/to/target/item' )

# update a link target
cnt.Create( id='/path/to/item', targetId='/path/to/anotherTarget' )

# instanciate a template
# (there is not really a template, you just 'copy' some reference branch
# and optionally change the target name)
cnt.Copy( item=ShotTemplate, under=SeqItem, name='P002' )

# build a view
view = cnt.GetView(
    root=LIB.id,
    select='CHARS/*',
    where='@TASKS/*/STATUS == "WIP"',
    attrs=['hasHair', 'hasCFX', 'hasSFX'],
    sortBy='@../', # ../ is the family
)
dictList
#-#######################
#--- Use Cases
#-#######################

# Print the edit and maya ranges for all active shots 
# in a known sequence, ordered by edit start frame
cnt = Connection( serverPath, login )
shots = cnt['PE/S0100/P*':'@.active']
for shot in shot << '@RANGES/EDIT.RecIn':
    edit = shot['RANGES/EDIT']
    anim = shot['RANGES/ANIM']
    print edit.RecIn, edit.RecOut, anim.AnimStart, anim.AnimEnd


# Add a Task to all shots with the Character KYLE
cnt = Connection( serverPath, login )
taskTemplate = cnt['TEMPLATES/TASKS/HAIR']
shotTasks = cnt['PE/S*/P*/TASKS':'"KYLE" in @../CASTING/CHARS']
[ cnt.Copy( taskTemplate, taskGrp ) for taskGrp in shotTasks ]









